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TESTING  REMEMBER- LEVEL  OBJECTIVES 


Instructions 


For  this  exercise,  you  will  write  test  items  for  the  list  of 
20  objectives  given  below.  The  exercise  has  two  parts.  The  first 
part  requires  you  to  write  test  items  that  are  consistent  with 
the  objectives  and  the  second  asks  you  to  write  inconsistent  test 
items . 

For  the  first  part,  objectives  1  through  11,  all  items  are 
constructed-response .  For  the  second  part,  objectives  12  through 
20,  the  type  of  item(s)  to  be  written  is  given  immediately 
following  the  objective. 

The  second  part  is  designed  to  help  you  deal  with 
constraints  on  testing.  These  items  types  should  be  used  only 
when  absolutely  necessary.  For  multiple-choice  items  in  the 
second  part  that  require  subject  matter  expert  knowledge  to 
develop  alternatives,  do  not  write  the  alternatives?  instead, 
just  indicate  a.  b.  c.  etc.  If  the  objective  calls  for  a  series 
of  multiple-choice  items,  just  write  one  prototype  item. 

Remember,  before  you  try  to  write  the  test  item,  you  should 
classify  the  objective  and  then  look  up  the  appropriate  item 
format  in  the  handbook. 

If  paraphrasing  would  be  appropriate,  make  a  note  of  it  when 
you  write  the  item. 


Consistent  Test  Items 

1.  The  student  will  write  from  memory  th-  or  .  i  ■ ;  i  ■ '  ■  ■: 

operation  of  a  rotary-gear  pump  as  lescribei  m  Teen.  M  in 1 1 

2.  The  student  will  recall  in  writim  wn  it  will  happen  i  f  : 
oil  in  the  landing  gear  of  an  F-18  aircraft  As  '  ant  m  in  ■:  ‘ 
insoluable  materials. 

1 .  The  student  will  write  from  memory  the  cor  rv  -t  se  -ju  >■-.  .  • 
steps  for  testing  common  transistors  using  a  Simps  in  260! 
mu l t imeter . 

4.  Given  a  diagram  of  the  NCR  Cash  Register ,  the  student  will 
label  the  parts. 

5.  The  student  will  write  from  memory  the  i.-roce  1  :r  ->  •  sett  i 
the  dwell  angle  in  a  GM  half-ton  truck. 

6.  The  student  will  recall  in  writing  the  names  v the  f-liawi  ; 

10  schematic  symbols:  (1)  ...,  (2)  ....  etc. 

7.  The  student  will  recall  in  writing  the  major  steps  from 
origination  to  passage  of  a  bill  in  the  U. S.  Congress ,  as  list e  i 
in  the  text. 

8.  The  student  will  recall  in  writing  the  functions  'f  ov'i  of 
the  control  knobs  on  the  SQQ89  sonar. 

9.  Given  a  picture  of  any  wrench,  the  student  will  onl  ly  ■  r  •  >  1 
the  name  of  the  wrench. 

10.  The  student  will  write  from  memory  the  Ohm's  Law  form  i  I  t 
(  I=E/R)  for  calculating  total  current  in  a  circuit. 

LI.  The  student  will  describe  the  radar  sigmi  p  i  r  ame  f  •„> : 
measurement  characteristics  of  SPS  and  PRF  that  are  used  in 
identifying  circular  sector,  conical,  and  steady  types  f  some 
on  an  analysis  scope  and  by  listening  to  audio  output. 
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Inconsistent  Test  Items 


These  item  formats  should  only  be  used  when  there  are 
constraints  on  testing.  Paraphrasing  should  be  used  when 
possible.  Vbu  should  be  familiar  with  the  sections  in  Chapter  3 
of  the  handbook  on  "Paraphrase  Test  Items,"  "Rules  and  Guidelines 
for  Writing  Test  Items,"  and  "Constraints  on  Testing"  before 
completing  this  exercise. 

12.  The  student  will  write  from  memory,  in  the  proper  order,  the 
procedures  performed  by  a  gun  captain  while  firing  during  general 
quarters.  Write  a  series  of  multiple-choice  items. 

13.  The  student  will  write  from  memory,  in  the  proper  order,  the 
steps  of  the  procedure  for  message  reception  and  duplicate 
checking  as  listed  in  the  current  edition  of  NTP-4.  Write  a 
series  of  multiple-choice  items. 

14.  Hie  student  will  label  on  a  diagram  all  the  components  of 
the  water  cooling  system  for  a  GM6-71  series  diesel  engine. 

Write  a  matching  item. 

15.  The  student  will  write  from  memory  the  six  pieces  of 
silverware  used  for  a  formal  dinner  setting.  The  silverware 
should  be  listed  in  order  from  left  to  right.  Write  a  matching 
item. 


16.  The  student  will  list  from  memory  the  four  types  of  messages 
that  may  be  broadcast  on  a  TADIL  net.  Write  a  series  of 
multiple-choice  items. 

17.  The  student  will  write  from  memory  the  functions  of  the 
following  objects  as  given  in  technical  manual  xxy:  metal 
fastener,  wrench,  reciprocating  pump,  reducing  valve,  worm  gear, 
and  micrometer.  Write  a  series  of  multiple-choice  items. 

18.  The  student  will  write  from  memory  the  names  of  the 
electronic  variables  represented  by  the  following  letters:  I,  E, 
and  R.  Write  a  matching  item. 

19.  The  student  will  write  from  memory  the  names  and 

characteristics  of  each  of  the  following  types  of  Navy  call 
signs:  international  U.S.  Navy  ship,  international  U.S.  Navy 

shore,  indefinite,  task  organization,  and  voice.  Write  a  series 
of  multiple-choice  items. 

20.  The  student  will  recall  orally  the  steps  of  the  procedure 
for  sharpening  a  pocket  knife  as  listed  in  NAVPERS  93438-1B. 

Write  a  matching  item. 
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Feedback  for  Cons i stent  Test  Items 

L.  In  the  space  below,  describe  the  principles  of  operation  :>f  : 

rotary-gear  pump.  Your  answer  should  include  all  the  nr  in  '  i  p  1 
liscussed  in  Tech.  Manual  XXX. 

2.  In  the  space  below,  explain  what  will  happen  to  the  l  an  i  \n  i 
gear  mechanism  of  an  F-18  aircraft  if  the  lubricating  oil  is 
contaminated  by  insoluable  materials. 

J.  In  the  space  below,  write  the  steps  for  testing  common 
transistors  using  a  Simpson  260P  multimeter,  in  the  correct 
order . 

4.  Label  the  diagram  of  the  NCR  cash  register  pictured  be!  aw. 

5.  In  the  space  below,  write  the  steps  for  setting  the  dwell 
angle  in  a  CM  half-ton  truck,  in  the  correct  order. 

6.  For  each  of  the  schematic  symbols  pictured  below,  write  the 
name  of  the  symbol  in  the  blank  next  to  the  symbol • 

7.  In  the  space  below/  write  the  major  steps  a  bill  takes  from 
origination  to  passage  to  get  through  the  U.S.  Congress.  Tae 
steps  must  be  written  in  the  correct  order. 

8.  Below  is  a  picture  of  the  control  panel  for  the  SQQ89  sonar. 
There  is  an  arrow  drawn  from  each  of  the  control  knobs  to  a  hi ank 
line.  Write  the  function  of  each  knob  on  blank  line  for  that 
knob . 


9.  When  I  point  to  a  picture  of  a  wrench  on  this  poster,  I  we.t 
you  to  tell  me  the  name  of  the  wrench. 

10.  Write  the  Ohm's  Law  formula  for  calculating  total  current  in 
a  circuit. 

11.  In  the  space  below,  describe  the  radar  signal  parameter 
measurement  characteristics  of  SPS  and  PRF  that  are  use  1  in 
identifying  circular  sector,  conical,  and  steady  types  of  scans 
on  an  analysis  scope  and  by  listening  to  audio  output. 
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Feedback  for  Incons istent  Test  Items 

12.  The  form  for  a  series  of  multiple-choice  items  for  this 
objective  is  as  follows: 

Which  of  the  following  is  the  (step  number)  step  of  the  procedure 
performed  by  a  gun  captain  while  firing  during  general  quarters? 

3  •  •  •  • 

l)  t  •  a  • 

G  *  It* 

d .  ... 

The  alternatives  should  be  the  other  steps  of  the  procedure. 
Paraphrasing  should  be  used  when  possible. 

13.  The  form  for  a  series  of  multiple-choice  items  for  this 
objective  is  as  follows: 

Which  of  the  following  is  the  (step  number)  step  of  the  procedure 
for  message  reception  and  duplicate  checking  as  listed  in 
the  current  edition  of  NTP-4? 


a . 

b . 


c . 

d. 


The  alternatives  should  be  the  other  steps  of  the  procedure. 
Paraphrasing  should  be  used  when  possible. 

14.  Below  is  a  diagram  of  (346-71  series  diesel  engine.  Each 
component  of  the  cooling  systems  has  a  letter,  tou  must  match 
that  letter  with  the  appropriate  component  name.  The  component 
names  are  listed  on  the  right-hand  side  of  the  page.  A  blank  is 
provided  next  to  each  name.  Write  the  appropriate  letters  in  the 
appropriate  blanks. 

15.  Below  are  pictured  the  six  pieces  of  silverware  used  for  a 
formal  dinner.  Each  piece  is  numbered.  Match  each  number  with 
the  appropriate  name.  The  names  are  listed  on  the  right  side  of 
the  page.  Write  the  appropriate  number  in  the  blank  next  to  each 
name.  Next,  on  the  six  blank  lines  at  the  bottom  of  the  page, 
arrange  the  numbers  in  the  correct  order  that  the  silverware 
would  appear  on  the  table  from  left  to  right. 
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L6.  There  should  be  four  multiple-choice  items  for  this 
objective.  They  should  take  the  following  form: 


Which  of  the  message  types  listed  below  may  be  broadcast  on  s 
TADIL  net? 

a .  ... 

b .  ... 

c  •  •  •  » 

d .  ... 

The  alternatives  should  be  message  types  that  are  broadcast  on 
other  types  of  nets.  Paraphrasing  cannot  be  used  here. 

17.  The  form  for  a  series  of  multiple-choice  items  for  this 
objective  is  as  follows: 

Which  of  the  statements  below  describes  the  function  of 
the  (object  name)? 

a .  ... 

b .  ... 

c  •  •  •  • 

d.  ... 

The  alternatives  should  be  the  functions  of  the  other  objects. 
Paraphrasing  should  be  used  when  possible. 

13.  Match  the  letters  in  column  A  with  the  electronic  variable 
names  in  column  B. 

19.  The  multiple-choice  items  for  this  objective  should  have  the 
following  form: 

Which  of  the  sets  of  characteristics  given  below  are  the 
characteristics  of  the  (Call  Sign  name)? 

a.  charac.  a,  charac.  b,  charac.  c 

b.  charac.  d,  charac.  e,  charac.  f 

c.  charac.  g,  charac.  h,  charac.  i 

d.  charac.  j,  charac.  k,  charac.  1 

Paraphrasing  cannot  be  used. 

20.  Match  each  of  the  steps  for  sharpening  a  pocket  knife  listed 
below  with  the  aopropriate  number  for  that  step.  Write  the 
correct  step  number  in  the  blank  next  to  each  step.  To  be 
correct,  you  must  use  the  order  of  steps  given  in  NAVPERS 
92438-1B. 
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Instruct  ions 


TESTING  A  USE-PROCEDURE  OBJECTIVE 


In  this  exercise,  you  will  be  given  a  USE-PROCEDURE 
objective.  tour  task  is  to  use  the  guidelines  in  "Rules  for 
Designing  Test  Items  for  USE-PROCEDURE  Objectives"  (Chapter  4  of 
the  handbook,  page  75).  tou  should  (1)  determine  whether  a 
product,  process,  or  both  should  be  evaluated,  (2)  determine 
whether  a  simulation  should  be  used,  (3)  develop  the 
product/process  measurements— including  actual  checklists/rating 
scales  and  the  directions  for  their  administration,  (4)  decide 
how  many  times  the  procedure  should  be  performed,  and  (5)  develop 
any  REM  EMBER- level  test  items  that  need  to  be  tested  during  the 
USE-PROCEDURE  test. 

The  information  needed  to  do  this  is  included  here  and  in 
the  handbook.  Look  at  the  main  ideas  to  be  considered  under 
"Testing  USE-PROCEDURE  Objectives"  (page  75)  before  you  begin 
item  development. 

Exercise 

Objective 

Given  a  sharp  knife  (use  a  felt  tip  pen  to  simulate  a  sharp 
knife),  matches,  and  a  scarf,  the  student  will  demonstrate 
first  aid  for  a  simulated  snakebite  when  the  victim  develops 
severe  symptoms,  is  breathing,  and  has  a  pulse.  The  nearest 
hospital  is  over  five  hours  away.  Simulate  the  cuts  across 
the  bite  with  a  felt-tip  pen. 

First  Aid  for  Snakebite 


The  following  information  is  taken  from  the  American  Red 
Cross  pamphlet  (1980)  on  "first  aid  for  snakebite." 


First  Aid  for  Snakebite 

The  victim  of  a  snakebite  must  have  prompt  medical  at¬ 
tention.  The  most  important  step  is  to  get  the 
snakebite  victim  to  the  hospital  qu ickly .  Meanwhile, 
take  the  fol lowing  first  aid  measures: 

1.  Keep  the  victim  from  moving  around. 

2.  Keep  the  victim  as  calm  as  possible  and  preferably 
in  a  lying  position. 

3.  Immobilize  the  bitten  extremity  and  keep  it  at  or 
below  heart  level. 
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If  the  victim  can  reach  a  hospital  within  4  or  5  hoars 
and  if  no  further  symptoms  develop,  no  further  first 
aid  measures  need  be  applied. 


4.  If  mi Id-to-moderate  symptoms  develop,  apply  a  con¬ 
stricting  band  2  to  4  inches  above  the  bite,  but  not 
around  the  head,  neck,  joints,  or  trunk.  TVie  ba.nl 
should  be  between  the  bite  and  the  heart. 

The  band  should  be  3/4  to  1-1/2  inches  wile,  not  tnm 
like  a  rubber  band.  Tne  band  should  be  snug  but  loose 
enough  for  a  finger  to  be  slipped  underneath.  Watcn 
for  swelling.  Loosen  the  band  if  it  becomes  too  tight, 
but  do  not  remove  it.  Periodically  check  the  pulse  in 
the  extremity  beyond  the  bite  to  ensure  that  the  blood 
flow  has  not  stopped. 

5.  If  severe  symptoms  develop,  keep  the  victim  lying 
down  and  comfortable  and  maintain  his  body  temperature. 
If  breathing  stops,  give  mouth- to-mouth  resuscitation. 
If  there  is  no  pulse,  perform  cardiopulmonary  resusci¬ 
tation  (CPR)--if  you  have  been  trained  to  do  so. 

As  soon  as  possible  make  an  incision  over  each  fang 
mark  and  apply  suction  immediately.  Apply  a  constrict¬ 
ing  band  (see  Step  4)  if  that  has  not  already  been 
done,  and  make  cuts  in  the  skin  through  the  fang  marks. 
Use  a  sharp,  sterilized  knife.  Matches  can  be  used  to 
sterilize  the  knife.  Cuts  should  be  no  deeper  than  just 
through  the  skin  and  should  be  1/2  inch  long,  extending 
over  the  suspected  venom  deposit  point.  (Because  a 
snake  strikes  downward,  the  deposit  point  is  usually 
lower  than  the  fang  mark.)  Cuts  should  be  made  along 
the  long  axis  of  the  limb.  Do  not  make  cross-cut 
incisions .  Do  not  make  cuts  on  the  head ,  neck,  or 
trunk  or  joints . 

Apply  suction  with  a  suction  cup  for  30  minutes.  If  a 
suction  cup  is  not  available,  use  your  mouth.  There  is 
little  risk  to  the  rescuer  who  uses  his  moutn,  but  it 
is  recommended  that  the  venom  not  be  swallowed  and  that 
the  mouth  be  rinsed  out. 

If  the  hospital  is  not  close--that  is,  if  it  cannot  be 
reached  in  4  or  5  hours--take  the  following  measures: 

1.  Keep  trying  to  obtain  professional  care,  either  by 
transporting  the  victim  to  a  place  where  medical  care 
is  available  or  by  using  an  emergency  communications 
system  to  obtain  medical  adyice. 


1 


2.  If  no  symptoms  develop,  Keep  trying  to  reach  the 
hospital  and  give  the  general  first  aid  described  above 
in  steps  1,  2,  and  3. 

3.  If  any  symptoms  at  all  develop,  apply  a  constricting 
band,  make  incisions,  and  apply  suction  immediately,  as 
described  above  in  steps  4  and  5. 

Use  the  above  information  and  the  handbook  with  the  steps 
below  to  develop  an  evaluation  of  the  procedure  and  any  necessary 
testing  of  REMEMBER-level  items. 

Step  1.  Determine  TASK  LEVEL  and  CONTENT  TYPE.  Before  you 
start  anything  you  should  classify  the  objective  and  determine  if 
you  are  in  the  right  chapter. 

Steps  2  and  3.  Determine  if  the  Cfc>  j  ective  Spec i f ies  a 
Product  and  Determine  Whether  to  Test  Product ,  Process,  or  Both. 
Refer  back  to  the  objective  to  decide  if  a  tangible  product  or 
process  or  both  result  from  the  procedure.  Steps  2  and  3  are 
related  to  the  product/process  decision  about  what  should  be 
evaluated.  Make  some  notes  about  what  you  decide. 

Step  4.  Determine  Whether  Simulation  is  Necessary  or 
Desirable.  Here  the  objective  already  specifies  a  simulated 
demonstration . 

Steps  5  and  7.  Analyze  the  Process  and  Determine  Whether  to 
Use  Checklists  and/or  Rating  Scales .  Look  at  the  Red  Cross 
information  on  what  to  do  if  severe  symptoms  develop.  Analyze 
the  procedure  into  discrete  steps.  Decide  whether  a  checklist  or 
rating  scale  is  the  most  appropriate  measurement  method  for  each 
step . 


Step  6 .  Analyze  the  Product .  N/ A 

Steps  8  and  9.  Develop  Checklists  and  Develop  Rating 
Scales .  After  the  steps  are  specified,  decide  which  measurement 
method — a  rating  scale  or  a  checkl ist--can  be  used  best  to 
evaluate  that  step  and  then  develop  the  lists  or  scales.  For 
purposes  of  this  exercise,  only  develop  checklists  and/or  rating 
scales  for  the  section  dealing  with  the  constricting  band. 

Step  1 0 .  Develop  Direct  ions  for  Administering  the  Rating 
Scales  and7or  Checkl ists .  Make  some  notes  about  how  the 
evaluator  can  use  the  forms  for  diagnosis  of  problem  areas  and 
how  they  should  be  scored. 

Step  1 1 .  Fie  Id  Test  and  Revise  the  Rating  Scales  and/or 
Checklists.  (No  need  to  worry  about  this  here — vou'll  aet  It 
later  71 - 
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Step  12 .  Determine  How  Many  Repetitions  of  the  Procedure 
ar e  Required .  Think  about  the  complexity  of:  the  procedure^ 

Step  1 3 .  Develop  REM EMB ER- level  Tes  t  Items  if  Necessary. 

In  this  example,  there  may  be  REMEMBER-level  questions  you  will 
want  to  ask  before  the  demonstration  begins.  This  is  especially 
important  if  there  are  possible  safety  hazards  involved.  Review 
the  Red  Cross  material  and  make  some  notes  about  what  REMEMBER- 
level  information  might  be  tested  during  the  USE-PROCEDURE  test. 


Feedback 


Step  1.  Determine  TASK  LEVEL  and  CONTENT  TYPE.  'mis 
objective  is  a  USE-UNAIDED  PROCEDURE. 

Steps  2  and  3.  Determine  i f  the  Objective  Spec i f ies  a 
Product  and  Determine  Whether  to  Test  Product ,  Process,  or  Both . 
We  could  measure  the  degree  of  crippling  injury  to  the  victim  as 
a  product,  but  that  is  unreasonable-  me  objective  specifies 
that  a  first  aid  demonstration  is  the  action  to  be  measured, 
mis  is  a  process.  Therefore,  in  this  objective,  we  are  more 
interested  in  testing  the  process;  that  is,  the  steps  the 
students  perform  as  they  treat  the  victim. 

Step  5.  Determine  Whether  Simulation  is  Necessary  or 
Desirable.  mis  was  specified  in  the  objective. 

Step  5.  Analyze  the  Process .  Notice  that  the  objective 
states  the  victim  has  a  severe  reaction  to  a  snakebite.  All 
cases  of  snakebite  should  be  treated  as  described  under  "Give 
first  aid,"  including  those  where  there  is  a  severe  reaction.  In 
this  case,  we  are  assuming  the  closest  hospital  is  over  five 
hours  away.  me  patient  is  breathing  and  has  a  pulse.  Below  is 
a  sample  of  how  the  procedure  may  be  broken  into  measureable 
steps : 

Give  first  aid  by — 

First,  keeping  the  victim  lying  down  and  maintain  his  body 

temperature . 

Second,  immobilizing  the  bitten  limb  at  or  below  heart 

level . 

1.  Apply  a  constricting  band. 

a.  Choose  a  band  that  is  3/4  to  1-1/2  inches  wide,  not 
as  thin  as  a  shoe  string. 

b.  Apply  the  constricting  band  2  to  4  inches  above  the 
bite,  towards  the  heart,  and  loose  enough  to  slip  a 
finger  under. 

c.  Do  not  apply  around  a  joint,  or  the  head,  neck,  or 
trunk . 

d.  Loosen  the  band  if  swelling  occurs,  but  do  not 
remove  it. 

e.  Check  the  pulse  beyond  the  bite  for  blood  flow. 
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2.  Make  an  incision. 


a.  Sterilize  a  sharp  knife. 

b.  Make  1/2-inch  long,  shallow  cuts,  no  deeper  than 
just  through  the  skin,  extending  over  the  fang  marks 
and  the  suspected  venom  deposit  (usually  lower  than  the 
fang  marks ) . 

c.  Cuts  should  be  made  lengthwise  along  the  limb. 

d.  Do  not  make  cross-cut  incisions.  Do  not  make  cuts 
on  the  head,  neck,  joints,  or  trunk. 

3.  Apply  suction. 

a.  Periodically  check  the  pulse  in  the  limb  beyond  the 
bite  to  be  sure  blood  flow  has  not  stopped. 

b.  Apply  suction  to  the  bite  for  30  minutes  with  a 
suction  cup  or  your  mouth. 

4.  Get  to  the  hospital  as  soon  as  possible. 

Most  of  the  steps  are  explicit  enough,  but  step  2  requires 
more  explanation  about  how  to  sterilize  a  knife.  Also,  you  may 
wai co  consider  simulating  this  step,  in  order  to  save  time. 

Step  7.  Determine  Whether  to  Use  Checkl ists  and/or  Rating 
Scales .  The  steps  under  "Give  first  aid"  can  be  measured  with  a 
checklist  because  they  are  simply  performed  or  not.  All  of  step 
1  can  be  measured  with  a  checklist.  Step  2b  and  2c  refer  to  the 
cuts  and  require  a  rating  scale.  Steps  2a  and  2d  require  a 
checklist.  Step  3a  can  be  a  checklist.  Step  3b  requires  a  rating 
scale . 
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Steps  8  and  9 .  Develop  Checklist  and  Develop  Rating  Scales . 
As  an  example,  the  checklist  For  step  1,  applying  the 
constricting  band,  is  developed.  Note  that  characteristics  that 
should  be  absent  are  stated  negatively  on  the  checklist. 

Constricting  band:  yes  no 

la.  Band  is  between  3/4  to 

1  1/2  inches  wide.  .  . 

lb.  Band  is  2  to  4  inches 
above  bite  and  towards 

the  heart.  .  . 

Band  is  loose  enough 
to  slip  one  finger  under 

but  not  two .  .  . 

Id.  If  swelling  occurs, 

loosen  band.  .  . 


Do  not  remove  band .  .  . .  . 

le.  Check  for  pulse  beyond  the  bite.  .  . 

Notice  that  steps  lb  and  Id  have  two  parts.  This  is  because 
these  steps  have  more  than  one  action.  These  steps  could  have 
been  broken  down  into  smaller  steps. 

Step  10.  Develop  Directions  for  Administering  the  Rating 
Scales  and7or  Checklists .  Some  of  the  things  you  may  have  noted 
about  evaluating  a  procedure  are  that  the  steps  provide  an 
obvious  indication  of  where  there  are  problems,  that  those 
problems  are  usually  due  to  memory  lapses,  and  that  diagnosis  can 
be  specific  and  feedback  immediate.  look  at  the  directions  that 
are  given  on  page  86  of  the  handbook.  Something  like  those 
should  be  developed  for  each  set  of  scales/checklists. 

In  this  case,  the  student  should  receive  supervised 
remediation  for  each  error  until  he  is  able  to  perform  accurately 
on  a  retest  without  help  from  the  evaluator. 

Step  11 .  Field  Test  and  Revise  the  Rating  Scales  and/or 
Oiecklists .  Go  to  Chapter  6  of  the  handbook  for  the  section  on 
'*  Determining  Reliability  of  Checklists  and  Rating  Scales." 


Step  12 .  Determine  How  Many  Repetitions  of  the  Procedure 
are  Required .  Based  on  the  complexity  of  this  objective,  it  was 
decided  that  the  student  must  perform  the  entire  procedure 
correctly  one  time.  (This  is  the  kind  of  decision  that  can  only 
be  made  with  the  help  of  a  subject  matter  expert.) 

Step  13 .  Develop  REM  EMBER- level  Test  Items  if  Necessary. 
Notice  that  there  are  some  actions  in  the  procedure  that  the 
rescuer  must  remember  NOT  to  do.  They  could  be  more  dangerous  to 
the  victim  than  the  snakebite.  For  example,  the  constricting  band 
should  not  be  applied  around  a  joint  or  the  head,  neck,  or  trunk. 
Also,  cuts  should  not  be  made  on  the  head,  neck,  or  trunk,  and 
cross-cuts  should  not  be  made. 

Several  REMEMBER-level  test  items  may  be  given  here  because 
of  the  potential  safety  hazard  to  the  individual  who  is  the 
victim.  The  test  items  should  be  given  orally  and  the  procedures 
described  in  Chapter  3  should  be  used  to  develop  the  items. 


Instruct  ions 


TESTING  A  USE- CATEGORY  OBJECTIVE 


In  this  exercise,  you  will  be  given  a  USE-CATEGORY 
objective,  tour  task  is  to  use  the  guidelines  in  the  "Testing 
USE-CATEGORY  Objectives"  section  of  the  handbook  to  analyze  the 
objective,  determine  item  formats,  construct  actual  items,  and 
develop  standards  and  instructions  for  scoring.  All  the 
information  needed  for  these  steps  will  be  provided. 

Exercise 

Objective 

Given  a  description  of  the  characteristics  of  a  USN  guided 
missile  destroyer,  the  student  will  write  which  of  the  four 
classes  of  guided  missile  destroyers  the  ship  belongs  to; 
KIDD,  ADAMS,  COONTZ,  SHERMAN-HULL  or  that  the  ship  is  not  a 
USN  guided  missile  destroyer. 

Step  1.  Analyze  the  Objective  to  Determine  the  Test  Item 
Domain 

This  section  contains  the  information  you  will  need  to 
analyze  the  objective. 

The  Navy  has  four  classes  of  guided  missile  destroyers 
( DDG) :  KIDD  Class,  ADAMS  Class,  COONTZ  Class,  ana  SHERMAN-HULL 
Class.  The  KIDD  and  COONTZ  class  ships  carry  Standard-ER 
missiles  and  the  other  two  classes  carry  Tartars.  KIDD  and  ADAMS 
class  ships  have  two  5-inch  guns  while  the  other  two  classes  have 
only  one.  KIDD  class  ships  are  powered  by  two  gas  turbine 
engines;  the  other  three  classes  are  driven  by  two  geared  steam 
turbine  engines.  Finally,  SHERMAN-HULL  and  ADAMS  class  ships 
have  a  top  speed  of  30  knots,  and  the  other  two  classes,  33 
knots . 

Step  1A.  List  the  Critical  Character  is tics  that  Determine 
Category  Membership.  Use  the  information  in  the  paragraph  above 
to  develop  a  list  of  the  critical  characteristics  for  each  class 
of  destroyer. 

Step  IB.  List  exceptions  ( i f  any) .  In  this  example,  there 
are  no  exceptions. 
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Step  1C.  List  variable  characteristics  and  the  Values  These 
May  Take .  The  major  variable  characteristics  of  the  four  classes 
are : 

1.  Type  of  radar :  Ships  within  a  class  can  carry  different 
types  of  radars;  for  example  some  ADAMS  class  destroyers  carry 
SPS  39s,  while  others  carry  SPS  10s  and  SPS  40s.  This  is  also 
true  of  the  other  classes.  Therefore,  if  you  know  the  radar  a 
ship  carries,  you  can't  tell  the  class  because  there  is  at  least 
one  ship  in  each  class  with  that  type  of  radar. 

2.  Number  of  digits  in  the  hull  number :  All  classes  have 
ships  with  one-  and  two-digit  hull  numbers. 

Step  1 D.  Use  the  Critical  and  Variable  Character ist ics  to 
Determine  the  Diagnostic  Requirements  and  Specify  the  Types  of 
Items  Needed .  For  this  step  specify  a  complete  set  of  item 
types.  Use  the  chart  procedure  described  in  the  handbook.  Do 
not  include  type  of  radar  as  a  variable  characteristic;  however, 
do  include  hull  number.  Make  some  notes  about  how  radar  might  be 
included . 

Step  2.  Determine  item  formats 

Try  to  think  about  the  "job"  and  make  a  decision  about  an 
appropriate  item  format. 

Step  3.  Construct  Actua 1  Test  Items 

Assume  the  the  item  types  specified  in  step  one  are  enough. 

Step  4.  Develop  Standards  and  Instructions  for  Scoring  and 

Diagnosis 

Make  some  notes  about  the  standards  you  would  set  for  the 
items  you  developed  and  about  the  kinds  of  instructions  you  would 
provide  for  scoring  and  diagnosis. 

Step  5.  Develop  REMEMBER-leve 1  Test  Items  i f  Necessary 

Since  this  is  a  written  test,  we  will  assume  that  REMEMBER- 
level  items  have  already  been  tested. 
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Feedback 


Step  1A .  List  critical  characteristics : 

The  character istics  of  KIDD  class  destroyers  are: 

1.  Missiles:  SAM  Standard-ER 

2.  Guns:  Two  5-inch 

3.  Main  Engines:  TWo  gas  turbines 

4.  Speed:  33  knots 

The  characteristics  of  ADAMS  class  destroyers  are: 

1.  Missiles:  SAM  Tartar 

2.  Guns:  TWo  5-inch 

3.  Main  Engines:  TWo  geared  steam  turbines 

4.  Speed:  30  knots 

The  characteristics  of  COONTZ  class  destroyers  are: 

1.  Missiles:  SAM  Standard-ER 

2.  (kins:  One  5-inch 

3.  Main  Engines:  TWo  geared  steam  turbines 

4.  Speed:  33  knots 

The  characteristics  of  SHERMAN-HULL  class  destroyers  are: 

1.  Missiles:  SAM  Tartar 

2.  (kins:  One  5-inch 

3.  Main  Engines:  TWo  geared  steam  turbines 

4.  Speed:  30  knots 

Step  IB.  List  exceptions  ( i f  any) .  There  are  no 
exceptions. 


Step  1C.  List  variable  characteristics  and  the  Values  These 
May  Take.  No  feedback  required. 
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Step  ID.  Use  the  Critical  and  Variable  Characteristics  to 
Determine  the  Diagnost ic  Requirements  and  Specify  the  Types  of 
Items  Needed.  In  this  situation,  we  are  dealing  with  four 
classes  o£  ships,  some  of  which  have  the  same  characteristics. 
Therefore,  we  will  specify  the  item  types  for  each  category  one 
at  a  time,  examine  the  items  to  eliminate  duplications  caused  by 
classes  having  the  same  characteristics,  and  finally  specify 
additional  items  for  the  variable  characteristics. 

Items  for  KIDD  Class:  This  class  has  four  critical 
characteristics,  so  we  will  need  five  sample  (n  +  1)  items.  We 
will  need  one  item  that  has  Standard-ER  missiles,  two  5-inch 
guns,  two  gas  turbine  engines,  and  a  speed  of  33  knots--this  is 
our  true  example.  Next,  we  will  need  an  item  that  has  different 
missiles,  one  that  has  a  different  number  of  guns,  one  that  has 
different  engines,  and  one  that  has  a  different  speed. 

The  item  samples  required  for  the  other  classes  of  ships 
follow  the  same  pattern  as  those  required  for  the  KIDD  class.  A 
chart  for  KIDD  class  ships  would  look  like  the  following: 


Chart  for  K IDD  Clas s  Ships 


Hull  # 
41 


32 

3 


14 

5 


Guns 


x 

X 


Speed 


30 


33 

x 


Engines 
Gas  I  Steam 


Missiles 


ERs 

x 


X 

X 


Tartar 


This  chart  can  be  used  for  all  the  ship  classes  because  they 
all  have  the  same  critical  characteristics. 
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The  information  from  the  chart  can  be  turned  into  a  pool  of 
items.  Below  are  20  items. 

K  IDD 

Ship  41  Standard-ER,  two  5-inch  guns,  two  gas  turbines,  33  knots 
Ship  32  Tartar,  two  5-inch  guns,  two  gas  turbines,  33  knots 
Ship  3  Standard-ER,  one  5-inch  gun,  two  gas  turbines,  33  knots 
Ship  14  Standard-ER,  two  5-inch  guns, 

two  geared  steam  turbines,  33  knots 
Ship  5  Standard-ER,  two  5-inch  guns,  two  gas  turbines,  30  knots 


ADAMS 

Ship  66  Thrtar,  two  5-inch  guns,  two  geared  steam  turbines,  30  knots 

Ship  37  Standard-ER,  two  5-inch  guns, 

two  geared  steam  turbines,  30  knots 
Ship  58  Tartar,  one  5-inch  gun,  two  geared  steam  turbines,  30  knots 

Ship  9  Tartar,  two  5-inch  guns,  two  gas  turbines,  30  knots 

Ship  18  Tartar,  two  5-inch  guns,  two  geared  steam  turbines,  33  knots 

COONTZ 

Ship  11  Standard-ER,  one  5-inch  gun, 

two  geared  steam  turbines,  33  knots 
Ship  2  Tartar,  one  5-inch  gun,  two  geared  steam  turbines,  33  knots 
Ship  33  Standard-ER,  two  5-inch  guns, 

two  geared  steam  turbines,  33  knots 
Ship  74  Standard-ER,  one  5-inch  gun,  two  gas  turbines,  33  knots 
Ship  25  Standard-ER,  one  5-inch  gun, 

two  geared  steam  turbines,  30  knots 


SHERMAN- HULL 

Ship  6  Tartar,  one  5-inch  gun,  two  geared  steam  turbines,  30  knots 
Ship  17  Standard-ER,  one  5-inch  gun, 

two  geared  steam  turbines,  30  knots 
Ship  48  Tartar,  two  5-inch  guns,  two  geared  steam  turbines,  30  knots 
Ship  1  Tartar,  one  5-inch  gun,  two  gas  turbines,  30  knots 
Ship  20  Tartar,  one  5-inch  gun,  two  geared  steam  turbines,  33  knots 

Notice  that  ships  14  and  33,  66  and  48,  58  and  6,  and  2  and 
20  are  identical;  therefore,  33,  48,  6,  and  20  can  be  deleted. 

The  variable  characteristics  listed  above  can  be  added  to 
each  list  of  ship  descriptions  so  that  all  relevant  values  of  the 
variable  characteristics  are  represented.  This  has  already  been 
done  for  the  hull  numbers  in  the  sample  above.  lb  complete  the 
listing  of  variable  characteristics,  each  ship  should  be  given  a 
radar,  taking  care  not  to  correlate  radar  type  with  class.  For 
example,  Ship  6  could  be  given  an  SPS  39  radar  in  addition  to  the 
characteristics  listed  above. 
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Step  2.  Determine  Item  Formats 

The  "job"  this  objective  supports  is  not  entirely  cLear. 
Presumably,  Navy  officers  are  supposed  to  be  able  to  identify 
what  class  a  ship  belongs  to  based  on  its  characteristics.  The 
officers  should  have  memorized  the  possible  classes  and  be  able 
to  give  the  correct  class  name  when  presented  with  a  set  of 
characteristics.  They  should  also  be  able  to  tell  when  a  set  of 
characteristics  does  not  describe  any  class  of  ship.  Therefore, 
the  item  format  should  be  f i 1 1-in-the-bl ank . 

Step  2*  Construct  Actua 1  Tes t  Items 

For  this  exercise,  we  will  assume  that  16  items  are  enough. 
Thus,  all  that  remains  to  be  done  is  to  put  the  sample  items  into 
the  correct  format?  that  is,  f i 1 1 - in- the-blank .  Write  one  or 
two  sample  items. 

Step  4.  Develop  Standards  and  Instruct  ions  for  Scoring  and 

Diagnosis 

The  implied  standard  in  the  objective  is  "with  no  errors." 
This  will  be  the  standard  used  for  this  exercise. 

A  diagnostic  plan  must  be  developed  so  that  remediation  can 
be  accurately  described.  Possible  diagnoses  for  each  possible 
answer  for  each  item  should  be  identified.  Below  are  diagnosis 
for  two  of  the  sample  items. 


Item  Response 


1.  Standard  ER,  KIDD 

one  5-inch  gun, 
two  steam  turbines, 

33  knots. 


ADAMS 


Possible  Diagnosis 


Student  has  KIDD  and  COONT2 
Class  confused.  May  know 
that  KIDD  has  Standard-ER 
Missile  and  a  speed  of  33 
knots  but  is  unsure  of 
type  of  engine  and  number 
of  guns.  Check  response 
to  KIDD  class  item  and 
other  iistractors  with 
Standard-ER  and  33  knots. 

Student  has  ADAMS  and  COONTZ 
Class  confused.  May  know  that 
ADAMS  has  steam  turbine  engines 
but  is  unsure  of  the  other 
characteristics.  Check  response 
to  ADAMS  class  item  and  other 
distractors  with  steam  turbine 
eng ines . 
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COONTZ 


Correct . 


SHERMAN-HULL  Student  has  SHERMAN-HULL  and 

COONTZ  Class  confused.  May  know 
that  SHERMAN-HULL  has  one  5-inch  gun 
and  steam  turbine  engines  but 
is  unsure  of  the  other  characteristics. 
Check  response  to  SHERMAN-HULL  class 
item  and  other  distractors  with  one 
5-inch  gun  and  steam  turbine  engines. 


Not  a  DDG 


2.  Tartar,  KIDD 

one  5-inch  gun, 
two  gas  turbines, 

30  knots. 

ADAMS 


COONTZ 


SHERMAN-HULL 


Student  is  probably  guessing  and 
needs  to  review  the  characteristics 
for  COONTZ  class  ships. 

Student  is  probably  focusing  on 
gas  turbine  characteristic  and 
believes  that  anything  with  a 
gas  turbine  is  a  KIDD  class  ship. 

Student  may  know  that  ADAMS  class 
ships  have  Tartar  missiles  and 
a  speed  of  30  knots  but  is  unsure 
of  the  other  characteristics. 

Check  response  to  ADAMS  class  item 
and  other  distractors  with  Tartar 
missiles  and  30  knots  speed. 

Student  may  be  focusing  on  the 
number  of  5-inch  guns.  Responses 
to  COONTZ  and  SHERMAN-HULL  items 
should  be  checked  as  well  as 
responses  to  other  distractors  with 
one  5-inch  gun. 

Student  may  know  all  character ist ics 
of  SHERMAN-HULL  except  engine  type. 
Responses  to  KIDD  class  item  and 
other  items  with  gas  turbines  should 
be  checked. 


Not  a  DDG  Correct. 

TVie  rest  of  the  items  would  have  similar  diagnostic 
information . 
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J 


1 


TESTING  A  USE-RULE  OBJECTIVE 


Instructions 

In  this  exercise,  you  wiLl  be  given  a  USE-RULE  Objective, 
tour  task  is  to  use  tJbe  guidelines  in  the  "Testing  USE-RULE 
Objectives"  section  of  the  handbook  to  analyze  the  objective, 
determine  item  formats,  construct  actual  items,  and  develop 
standards  and  instructions  for  scoring.  All  the  information  you 
need  is  given  below. 

Exercise 

Cb ject ive 


You  are  a  golfer  who  hits  a  7  iron  150  yards  under  "normal" 
conditions.  The  task  is  to  select  the  right  club  to  hit 
from  150  yards  out  under  a  variety  of  conditions.  The 
variables  that  must  be  considered  are  wind,  pin  placement, 
and  air  temperature. 

The  objective  is:  Given  an  approach  shot  of  150  yards,  the 
golfer  will  evaluate  wind  direction,  pin  placement,  and  air 
temperature  and  select  the  appropriate  iron.  For  the 
purposes  of  this  example,  it  is  assumed  that  the  golfer 
normally  hits  a  7  iron  150  yards. 

Step  .1.  Analyze  the  Objective  to  Determine  the  Tes t  Item 
Domain 

This  section  contains  the  information  you  will  need  to 
analyze  the  objective. 

Rule  Statement 


The  three  factors  are  cumulative.  After  you  have  evaluated 
the  first  factor,  you  must  take  the  club  you  have  selected 
and  apply  the  second  factor  to  it.  The  same  is  true  for  the 
third  factor. 

Wind :  If  the  wind  is  toward  you,  subtract  one  club;  if  there 
is  no  wind,  do  not  change  clubs;  if  the  wind  is  behind  you, 
add  one  club.  Next,  evaluate  pin  placement. 

Pin  Placement :  If  the  pin  is  at  the  back  of  the  green, 
subtract  one  club;  if  the  pin  is  in  the  middle,  do  not 
change  clubs;  if  the  pin  is  at  the  front,  add  one  club. 

Next,  evaluate  air  temperature. 

Air  temperature :  If  the  air  temperature  is  below  70  degrees 
fahrenheit,  subtract  one  club;  if  it  between  70  and  85 
degrees,  do  not  change  clubs;  if  it  is  above  85  degrees,  add 
one  club. 
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From  this  information,  you  should  be  able  to  develop  a  flow 
chart.  Mou  should  also  decide  how  to  test  the  action  boxes  and 
what  paths  through  the  flow  chart  should  be  tested. 

Step  2.  Determine  Item  Formats 

If  more  than  one  format  is  appropr iate/ 1 ist  all  appropriate 
formats . 

Step  3.  Construct  Actual  Test  Items 

Determine  how  many  paths  we  need  to  trace  through  the  flow 
chart  in  order  to  test  all  the  decisions  and  steps. 

Step  4.  Develop  Standards  and  Instruct  ions  for  Scoring  and 

Diagnos  is  ~~ 

Think  about  the  "job"  and  set  standards  accordingly.  Also, 
make  some  notes  about  scoring  instructions . 

Step  5.  Develop  REMEMBER- level  Test  Items  if  Necessary 

For  this  example,  assume  that  REMEMBER-level  items  have 
already  been  tested. 
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Feedback 


Steg  1.  Analyze  the  Objective  to  Determine  the  Test  Item 
Doma in 

A  flow  chart  for  this  task  is  shown  below: 


Club  =  7  Iron 
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It  should  be  noted  that  this  is  part  of  a  larger  rule  in 
which  golfers  must  make  club  selections  from  a  variety  of 
distances  under  a  variety  of  conditions. 

Step  2.  Determine  Item  formats 

It  is  appropriate  to  use  written  test  items  that  describe  a 
situation  and  require  the  student  to  make  a  decision.  Therefore, 
this  example  is  similar  to  the  fowl  example  on  page  114  of  the 
handbook . 

Given  that  written  test  items  will  be  used,  the  question  is 
whether  constructed-response  or  selected-response  test  items  will 
be  used.  If  you  "remember  the  job,"  you  will  realize  that  either 
type  can  be  used  if  they  are  constructed  properly.  For 
constructed-response  items,  descriptions  of  situations  could  be 
written  and  students  could  be  required  to  write  the  appropriate 
club  number.  For  selected-response  items,  the  most  appropriate 
format  is  matching  situation  descriptions  with  club  numbers. 

This  is  because  the  task  on  the  job  is  to  evaluate  the  situation 
and  select  a  club  from  among  the  13  or  14  clubs  in  the  bag.  A 
matching  item  should  include  all  the  clubs  in  the  bag  for 
potential  matches.  Note  that  standard  multiple-choice  items 
(i.e.,  those  with  four  or  five  alternatives)  would  not  work 
because  not  all  possible  clubs  could  be  included  as  distractors. 

Step  3.  Construct  Actual  Test  Items 

Three  paths  through  the  chart  will  test  all  the  decisions 
and  steps.  In  order  to  be  diagnostic,  responses  should  be 
recorded  after  each  step  for  each  path  through  the  chart.  Note 
that  one  possible  correct  answer  is  a  10  iron.  Students  should 
be  told  prior  to  testing  that  a  10  iron  is  called  a  pitching 
wedge . 

Step  4.  Develop  Standards  and  Instructions  for  Scoring  and 

Diagnosis 

Because  the  job  requires  accuracy,  the  standards  should  be 
completely  correct  on  all  items.  Diagnosis  is  similar  to 
diagnosis  for  category  tasks.  If  the  student  selects  a  club 
higher  or  lower  than  the  correct  club,  it  can  be  inferred  that 
the  decision  rule(s)  has/have  not  been  followed,  or  that  the 
student  is  guessing.  The  latter  would  be  especially  true  if  a 
club  not  in  the  150-yard  range  were  selected. 

Step  5.  Develop  REM  EMBER- level  Test  Items  if  Necessary 

For  the  purposes  of  this  example,  no  previous  REM  EMBER- level 
objectives  are  included. 
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Instructions 


TESTING  A  USE-PRINCIPLE  OBJECTIVE 


In  this  exercise,  you  will  be  given  a  USE-PRINCIPLE 
objective.  Your  task  is  to  use  the  guidelines  in  the  "Testing 
USE-PRINCIPLE  Objectives"  section  of  the  handbook  to  analyze  the 
objective,  determine  item  formats,  construct  actual  items,  and 
develop  standards  and  instructions  for  scoring.  All  the 
information  you  need  from  technical  publications  is  given  below. 

Exercise 

Objective 


Given  a  diagram  of  a  spring-loaded  reducing  valve,  the 
student  will  (1)  explain  how  the  valve  operates  to  maintain 
reduced  output  pressure,  (2)  predict  the  effects  of  clogged 
passages,  sticking  valve-piston  stems,  and  a  ruptured 
diaphragm  on  the  valve's  operation,  and  (3)  diagnose  the 
probable  reasons  for  lack  of  flow  or  excessive  output 
pressure . 

Step  1.  Ana lyze  the  Objective  to  Determine  the  Test  Item 
Domain 

This  section  contains  the  information  you  will  need  to 
analyze  the  objective. 

An  explanation  of  how  a  spring-loaded  reducing  valve  works 
and  two  diagrams  of  a  spring-loaded  reducing  valve  are  given  on 
the  following  page.  The  picture  on  the  left  shows  the  valve 
during  normal  operation:  high-pressure  fluid  enters  and  reduced 
pressure  fluid  exits.  The  picture  on  the  right  shows  the  valve 
when  outlet  pressure  increases.  The  purpose  of  the  valve  is  to 
control  or  hold  output  pressure  below  some  value. 

The  following  explanation  of  the  valve's  operation  is  taken 
from  NAVEDTRA  10524-E,  Machinist's  Mate  3  &  2  Rate  Training 
Manual . 
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"The  principal  parts  of  a  spring-loaded  reducing  valve  are:  (1) 
the  main  valve,  an  upward-seating  valve  which  has  a  piston  on  top 
of  its  stem?  (2)  an  upward-seating  auxiliary  (or  controlling) 
valve  [with  its  stem  attached  to];  (3)  a  controlling  diaphragm; 
and  (4)  an  adjusting  spring. 

"High  pressure  fluid  enters  the  valve  on  the  inlet  side  and  acts 
against  the  main  valve  disk,  tending  to  close  the  main  valve. 
However,  high-pressure  fluid  is  also  led  through  ports  to  the 
auxiliary  valve,  which  controls  the  admission  of  high-pressure 
fluid  to  the  top  of  the  main  valve  piston.  The  piston  has  a 
larger  surface  area  than  the  main  valve  disc;  therefore,  high- 
pressure  fluid  acting  on  the  top  of  the  main  valve  piston  will 
tend  to  open  the  main  valve,  and  so  allow  fluid  at  reduced 
pressure  to  flow  out  the  discharge  side. 

"But  what  makes  the  auxiliary  valve  open  to  allow  high-pressure 
fluid  to  get  to  the  top  of  the  main  valve  piston?  The 
controlling  diaphragm  exerts  a  downward  pressure  upon  the 
auxiliary  valve  stem,  and  this  tends  to  open  the  valve.  However, 
reduced-pressure  fluid  is  led  back  to  the  chamber  beneath  the 
diaphragm,  and  this  fluid  exerts  a  pressure  upward  on  the 
diaphragm,  which  tends  to  close  the  auxiliary  valve.  The 
position  of  the  auxiliary  valve,  therefore,  is  determined  by  the 
position  of  the  controlling  diaphragm. 

"The  position  of  the  diaphragm  at  any  given  moment  is  determined 
by  the  relative  strength  of  two  opposing  forces:  (1)  the 
downward  force  exerted  by  the  adjusting  spring,  and  (2)  the 
upward  force  which  is  exerted  on  the  underside  of  the  diaphragm 
by  the  reduced-pressure  fluid.  These  two  forces  are  continually 
seeking  to  reach  a  state  of  balance,  and,  because  of  this,  the 
discharge  pressure  of  fluid  is  kept  constant  as  long  as  the 
amount  of  fluid  is  kept  within  the  capacity  of  the  valve." 


The  first  step  is  to  use  the  description  above  to  construct 
item  types  for  the  explanation,  prediction,  and  diagnosis  tasks 
required  in  the  objective.  Use  the  methods  in  the  handbook  to 
construct  explanation,  prediction,  and  diagnosis  statements. 

Step  2.  Determine  Item  Formats 

For  this  exercise,  you  should  next  determine  what  item 
format(s)  to  use.  Make  some  choices  in  the  space  below, 
according  to  the  guidelines  in  the  handbook.  If  you  choose 
multiple-choice  items,  you  should  determine  whether  or  not  to  use 
"written  simulations." 

Step  3.  Construct  Actual  Test  Items 

For  purposes  of  this  exercise,  construct  a  few  items  for  the 
explanation,  prediction,  and  diagnosis  tasks  required  in  the 
objective.  \bu  do  not  have  to  build  complete  sets  of  items  for 
all  the  tasks;  just  do  enough  to  show  how  you  would  do  it. 

Step  4.  Develop  Standards  and  Instructions  for  Scoring  and 

Diagnosis 

Make  some  notes  about  the  standards  you  would  set  for  the 
items  you  developed  and  about  the  kinds  of  instructions  you  would 
provide  for  scoring  and  diagnosis. 

Step  5.  Develop  RB4B1BER- level  Test  Items  if  Necessary 

Since  this  is  a  written  test,  we  will  assume  that  earlier 
items  have  already  tested  part  names,  etc.  Ubu  don't  have  to  do 
it  here. 


Feedback 


Step  1 .  Analyze  the  objective . 

The  first  step  is  to  write  each  cause-effect  statement  for 
the  explanation  task.  This  is  done  below  simply  by  revising  the 
explanation  taken  from  the  manual: 

1.  High-pressure  fluid  enters  the  valve  on  the  inlet  side  and 
acts  against  the  main  valve  disk,  tending  to  close  the  main 
valve . 

2.  High-pressure  fluid  is  also  led  through  ports  to  the 
auxiliary  valve. 

3.  The  spring  and  controlling  diaphragm  above  the  auxiliary 
valve  push  down  on  the  valve  stem,  tending  to  open  the  auxiliary 
valve . 

4.  The  auxiliary  valve  controls  the  admission  of  high-pressure 
fluid  to  the  top  of  the  main  valve  piston. 

5.  The  piston  has  a  larger  surface  area  than  the  main  valve 
disc,  so  high-pressure  fluid  acting  on  the  top  of  the  main  valve 
piston  will  tend  to  open  the  main  valve. 

6.  This  will  allow  fluid  at  reduced  pressure  to  flow  out  the 
discharge  side. 

7.  This  reduced  pressure  fluid  is  also  led  through  ports  to  the 
underside  of  the  auxiliary  valve  diaphragm. 

8.  If  the  discharge  pressure  increases,  increased  pressure  is 
led  to  the  underside  of  the  diaphragm,  pushing  it  up. 

9.  When  the  diaphragm  rises,  this  closes  the  auxiliary  valve. 

10.  When  the  auxiliary  valve  closes,  this  reduces  pressure  to 
the  top  of  the  main  valve  piston. 

11.  High-pressure  fluid  from  the  inlet  side  tends  to  close  the 
main  valve. 

12.  When  the  main  valve  closes,  pressure  in  the  discharge  side 
is  reduced. 

Statements  1  through  12  above  explain  the  valve's  normal 
operation.  These  will  be  used  to  build  explanation  items. 

For  prediction  items,  we  need  to  assume  new  "boundary 
conditions,"  and  then  revise  the  explanation  above.  The 
objective  calls  for  predictions  for  "clogged  passages,  sticking 
valve  stems,  or  a  ruptured  diaphragm." 
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If  the  high-pressure  passage  from  the  inlet  side  of  the 
valve  to  the  auxiliary  valve  were  clogged,  the  explanation  would 
be : 


1.  High-pressure  fluid  enters  the  valve  on  the  inlet  side  and 
acts  against  the  main  valve  disk,  tending  to  close  the  main 
valve . 

2.  No  high-pressure  fluid  is  led  through  ports  to  the  auxiliary 
valve,  because  the  port  is  clogged. 

3.  The  spring  and  controlling  diaphragm  above  the  auxiliary 
valve  push  down  on  the  valve  stem,  tending  to  open  the  auxiliary 
valve . 

4.  No  high-pressure  fluid  flows  through  the  auxiliary  valve  to 
the  top  of  the  main  valve  piston. 

5.  Therefore,  the  main  valve  remains  closed. 


If  the  low-pressure  passage  from  the  discharge  side  of  the 
valve  to  the  bottom  of  the  diaphragm  were  clogged,  the 
explanation  would  be: 

1.  High-pressure  fluid  enters  the  valve  on  the  inlet  side  and 
acts  against  the  main  valve  disk,  tending  to  close  the  main 
valve . 

2.  High-pressure  fluid  is  also  led  through  ports  to  the 
auxiliary  valve. 

3.  The  spring  and  controlling  diaphragm  above  the  auxiliary 
valve  push  down  on  the  valve  stem,  tending  to  open  the  auxiliary 
valve . 

4.  The  auxiliary  valve  controls  the  admission  of  high-pressure 
fluid  to  the  top  of  the  main  valve  piston. 

5.  The  piston  has  a  larger  surface  area  than  the  main  valve 
disc,  so  high-pressure  fluid  acting  on  the  top  of  the  main  valve 
piston  will  tend  to  open  the  main  valve. 

6.  This  will  allow  fluid  at  reduced  pressure  to  flow  out  the 
discharge  side. 

7.  This  reduced  pressure  fluid  is  not  led  through  ports  to  the 
underside  of  the  auxiliary  valve  diaphragm,  because  the  ports  are 
clogged. 
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S.  If  the  discharge  pressure  increases,  increased  pressure  is 
not  led  to  the  underside  of  the  diaphragm,  so  it  is  not  pushed 
up . 

9.  The  auxiliary  valve  does  not  close. 

10.  The  main  valve  does  not  tend  to  close. 

11.  Since  the  main  valve  does  not  close,  pressure  in  the 
discharge  side  is  not  reduced. 

Similar  explanations  can  be  constructed  for  sticking  valve 
steins  and  for  the  ruptured  diaphragm.  These  other  explanations 
will  be  similar  to  the  two  given  above,  because  sticking  valve 
stems  will  prevent  the  valves  from  working  as  they  should,  and  a 
broken  diaphragm  will  prevent  the  auxiliary  valve  from  operating 
when  required. 

The  objective  also  requires  the  student  to  "diagnose 
probable  reasons  for  lack  of  flow  or  excessive  outlet  pressure." 
These  are  the  symptoms  the  student  will  deal  with,  so  we  don't 
have  to  identify  them.  (Note  that,  if  the  objective  was  less 
specific  and  just  said  "troubleshoot  the  valve,"  then  we  would 
have  to  consult  an  subject  matter  expert  to  identify  the  symptoms 
that  most  often  occur  or  are  most  critical.)  Tb  identify  the 
causes  for  the  symptoms  of  lack  of  flow  or  excessive  outlet 
pressure,  we  would  simply  trace  backwards  through  the 
explanations  above.  For  example,  lack  of  flow  is  caused  by  the 
main  valve  failing  to  open.  This  means  pressure  is  not  getting 
to  the  top  of  the  main  valve  piston,  or  that  pressure  is  there, 
but  the  piston  is  stuck.  Lack  of  pressure  at  the  top  of  the  main 
valve  piston  means  that  the  auxiliary  valve  is  stuck  closed,  or 
that  the  high  pressure  passage  to  the  auxiliary  valve  is  clogged. 

A  similar  process  is  followed  for  the  other 
symptom — excessive  outlet  pressure.  It  is  often  necessary  to 
consult  an  subject  matter  expert  to  determine  the  possible  causes 
for  each  observed  symptom. 
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Step  2. 


Determine  Item  Formats 


The  best  type  of  item  for  this  objective  is  a  one-on-one 
oral  test,  with  short-answer  items.  If  you  had  to  use  a  written 
test,  then  fill-in  items  could  be  used.  Multiple-choice  items 
are  also  acceptable  if  the  items  are  carefully  developed  but  are 
not  the  best  choice.  One  reason  constructed-response  tests  are 
most  appropriate  here  is  that  the  job  the  student  will  perform 
after  training  will  probably  not  consist  of  selected-response 
situations.  For  example,  valves  will  not  have  four  possible 
faults  listed  on  them.  Another  reason  is  that  multiple-choice 
items  tend  to  have  hints  to  other  items  in  them. 

If  you  did  select  multiple-choice  items,  written  simulations 
should  not  be  used.  The  reason  for  this  is  that  the  objective 
only  requires  the  student  to  give  probable  reasons  or  causes  for 
observed  symptoms,  not  to  perform  additional  diagnostic  tests. 
Therefore,  we  can't  really  ask  a  series  of  items  that  trace  out  a 
troubleshooting  process. 

Step  3.  Construct  Actual  Test  Items 

Ybur  task  here  was  to  construct  a  few  items  for  the 
explanation,  prediction,  and  diagnosis  tasks  required  in  the 
objective. 

If  you  chose  a  short-answer  oral  test  format,  you  would 
simply  have  items  like: 

"Explain  how  the  spring-loaded  reducing  valve  operates." 

\bu  would  also  have  to  give  clear  directions  to  the  student 
about  the  form  and  content  of  an  acceptable  answer . 

If  you  chose  a  fill-in  test  format,  you  would  turn  each  of 
the  explanation  statements  into  questions  with  key  words  left 
out : 


"1.  High-pressure  fluid  enters  the  valve  on  the  inlet  side, 
and  acts  against  the  main  valve  disc.  This  causes  the  valve 
to 


2.  High-pressure  fluid  is  also  led  through  ports  to  the 


(etc.)" 

If  you  chose  a  multiple-choice  written  test,  then  each  of 
the  explanation  statements  from  step  1  would  be  turned  into 
multiple-choice  items.  The  distractors  would  have  to  be 
developed  carefully: 


"1*  High-pressure  fluil  enters  the  valve  on  the  inlet  side 
and  acts  against  the  main  valve  disk;  this  causes  the  main 

valve  to 

a.  open. 

b.  close. 

c.  remain  in  the  same  position. 

2.  High-pressure  fluid  is  also  led  through  ports  to  the 

a.  outlet  side  of  the  valve. 

b.  underside  of  the  diaphragm. 

c.  auxiliary  valve. 

d.  underside  of  the  main-valve  piston. 

3.  The  spring  and  controlling  diaphragm  above  the  auxiliary 
valve  push  down  on  the  valve  stem,  tending  to 

a.  open  the  auxiliary  valve. 

b.  reduce  pressure  under  the  diaphragm. 

c.  close  the  main  valve. 

d.  close  the  auxiliary  valve." 

Prediction  items  would  be  constructed  similarly,  except  that 
you  would  use  the  explanations  constructed  for  changes  in  the 
initial  or  "boundary"  conditions; 

"1.  If  the  high-pressure  passage  from  the  inlet  side  of  the 
valve  to  the  auxiliary  valve  were  clogged,  the  main  valve 
would 

a.  remain  open. 

b.  remain  closed. 

c.  operate  normally." 
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The  diagnosis  items  are  constructed  simply  by  asking  for 
possible  causes  for  each  observed  symptom: 

"1.  The  spring-loaded  reducing  valve  fails  to  open,  even 
when  high-pressure  fluid  is  present  on  the  inlet  side  of  the 
valve.  Which  of  the  following  are  possible  causes  for  this 
failure?  (Choose  all  of  the  following  that  apply). 

a.  The  high-pressure  passage  to  the  auxiliary  valve  is 
clogged . 

b.  The  diaphragm  is  ruptured. 

c.  The  auxiliary  valve  stem  is  stuck  in  the  closed 
position . 

d.  The  main  valve  stem  is  stuck  in  the  closed 
pos ition . 

e.  The  auxiliary  valve  stem  is  stuck  in  the  open 
position . “ 

Note  that,  for  multiple-choice  items,  you  must  attempt  to 
make  the  incorrect  alternatives  diagnostic  so  that  they  can  help 
identify  students'  misconceptions. 

Step  4.  Set  Standards  and  Develop  Instructions  for  Scoring 
and  Diagnosis . 

The  implied  standard  in  the  original  objective  is  "100 
percent  correct,"  and  there  is  no  reason  to  set  a  lower  standard 
for  any  of  the  items  constructed  here. 

The  next  step  is  to  develop  instructions  for  scoring  and 
diagnosis.  Instructions  are  described  below  for  the  sample  items 
we  specified.  Hie  instructions  for  your  items  should  be  similar. 

If  you  chose  a  short-answer  oral  test  format,  you  would 
simply  have  items  like: 

"Explain  how  the  spring-loaded  reducing  valve  operates." 

Ybu  would  also  have  given  clear  directions  to  the  student 
about  the  form  and  content  of  an  acceptable  answer.  Instructions 
for  scoring  this  item  should  list  the  critical  points  in  the 
explanation  that  the  student  should  give  and  should  specify  how 
credit  is  given  for  each  of  them.  Errors  that  students 
frequently  make  should  be  listed  so  that  they  can  be  diagnosed. 
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If  you  chose  a  fill-in  test  format,  each  of  the  explanation 
statements  was  turned  into  a  question  with  a  key  word  left  out: 

"1.  High-pressure  fluid  enters  the  valve  on  the  inlet  side 
and  acts  against  the  main  valve  disc.  This  causes  the  valve 
to 


2.  High-pressure  fluid  is  also  led  through  ports  to  the 


( etc  . ) " 

For  each  of  these  questions,  correct  answers  (and  synonyms) 
should  be  given.  For  example,  for  item  1  above,  correct  answers 
are  "close,"  "lift,"  or  "shut."  Incorrect  answers  should  also  be 
listed,  together  with  diagnoses  for  their  occurrence.  For 
example,  for  item  1,  the  most  common  wrong  answer  would  be 
"open."  A  student  might  give  this  answer  if  he  did  not 
understand  the  valve,  or  if  he  was  thinking  ahead  in  the 
explanation.  This  could  be  checked  with  follow-up  questions. 

For  item  2  above,  the  correct  answer  is  "the  auxiliary  valve." 

The  most  likely  incorrect  answer  is  "the  top  of  the  main  valve 
piston."  High-pressure  fluid  does  reach  the  top  of  the  piston, 
but  only  after  it  passes  through  the  auxiliary  valve,  which  must 
be  open. 

If  you  chose  a  multiple-choice  written  test,  each  of  the 
explanation  statements  was  turned  into  a  multiple-choice  item. 

"1.  High-pressure  fluid  enters  the  valve  on  the  inlet  side 
and  acts  against  the  main  valve  disk?  this  causes  the  main 
valve  to 


a .  open . 

b.  close. 

c.  remain  in  the  same  position. 

2.  High-pressure  fluid  is  also  led  through  ports  to  the 

a.  outlet  side  of  the  valve. 

b.  underside  of  the  diaphragm. 

c.  auxiliary  valve. 

d.  underside  o  :  the  main-valve  piston. 

3.  The  spring  and  controlling  diaphragm  above  the  auxiliary 
valve  push  down  on  the  valve  stem,  tending  to 

a.  open  the  auxiliary  valve. 

b.  reduce  pressure  under  the  diaphragm. 

c.  close  the  main  valve. 

d.  close  the  auxiliary  valve." 
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Again,  for  these  items,  the  correct  answers  ace  noted  an. 5 
possible  misconcept ions  for  each  of  the  incorrect  alternatives 
are  identified,  just  as  we  did  above  for  the  fill-in  items. 

Instructions  for  scoring  prediction  items  would  be 
constructed  similarly. 

The  diagnosis  items  were  constructed  simply  by  asking  for 
possible  causes  for  each  observed  symptom.  Instructions  for 
scoring  and  diagnosis  should  again  give  the  correct  answers,  and 
should  give  possible  misconceptions  that  might  have  led  to  wrong 
answers.  For  example,  in  the  following  item,  alternatives  a,  c, 
and  d  are  correct: 

"1.  The  spring-loaded  reducing  valve  fails  to  open,  even 

when  high-pressure  fluid  is  present  on  the  inlet  side  of  the 

valve.  Which  of  the  following  are  possible  causes  for  this 

failure?  (Choose  all  of  the  following  that  apply). 

a.  The  high-pressure  passage  to  the  auxiliary  valve 
is  clogged. 

b.  Ihe  diaphragm  is  ruptured. 

c.  The  auxiliary  valve  is  stuck  in  the  closed 
position. 

d.  The  main  valve  is  stuck  in  the  closed  position. 

e.  The  auxiliary  valve  is  stuck  in  the  open  position." 

If  alternative  b  is  selected,  the  student  probably  thinks 
that  pressure  on  the  diaphragm  tends  to  open  the  auxiliary  valve, 
rather  than  close  it.  If  alternative  e  is  selected,  the  student 
does  not  know  that  high-pressure  fluid  from  the  auxiliary  valve 
opens  the  main  valve. 

Instructions  for  scoring  and  diagnosis  should  also  include 
remedial  actions  to  be  taken  for  incorrect  answers. 

Step  5.  Develop  REM EMBER- level  Items  if  Necessary 

Assume  that  all  REMEMBER-level  items  have  been  tested 
previously . 
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